Adaptive Multi-Modal Ensemble Network for Video Memorability Prediction

Video memorability prediction aims to quantify the credibility of being remembered according video content, which provides significant value in advertising design, social media recommendation, and other applications. However, main attributes that affect have not been determined so making design model more challenging. Therefore, this study, we analyze experimentally verify how select most impact factors predict memorability. Furthermore, a new framework, Adaptive Multi-modal Ensemble Network, based on chosen vital efficiently. Specifically, first conduct three memorability, i.e., temporal 3D information, spatial information semantics derived from video, image caption, respectively. Then, Network integrates individual base learners (i.e., ResNet3D, Deep Random Forest Multi-Layer Perception) into weighted ensemble framework score In addition, also an adaptive learning strategy update weights importance is predicted by rather than assigning manually. Finally, experiments public VideoMem dataset demonstrate proposed method competitive results high efficiency for prediction.